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This report evaluates the Secondary Developmental 
Reading Program, a component of the Ohio Disadvantaged Pupil Program 
Fund (DPPF), in terms -of the 1982-83 program objectives. Twelve 
project reading- teachers worked in eight Columbus senior high schools 
with 843 pupils scoring at or below the- 36th percentile in.reading 
achievement. A pilot project using Commodore PET computers for 
computer-assisted instruction and computer-managed instruction 
(CAI/CMI) served 261 of the students. The program stressed literacy 
survival skills and featured diagnostic testing, individualized and 
small group instruction, on-going pupil evaluation, and teacher 
inservice meetings. Student data (n«372) were collected from pupil 
census information, the Metropolitan Achievement Test, and hands-on 
computer reading tests. Teacher data came from three inservice 
evaluation and assessment instruments. In-process evaluation data 
were obtained from classroom observations. The evaluation procedures, 
results and recommendations are discussed, including a cost-benefit 
analysis. The reading achievement gain objective of 10.5 Normal Curve 
Equivalents (NCE's)) was not attained, but the CAI/CMI objective of 
seven prescriptive* reading skills was. Only one of three inservice 
objectives was met. Due to ongoing problems of pupil attendance and 
achievement and drastic funding reductions, all DPPF project 
evaluations should be reviewed for progran revision. Appendix 
contains the evaluation and assessment forms. (BS) 
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Ohio Disadvantaged Pupil Program Fund 

FINAL .EVALUATION REPORT ' \ 

)- LANGUAGE DEVELOPMENT COMPONENT . ) 

SECONDARY DEVELOPMENTAL READING PROGRAM 

' July, 1,983 

* » , 

\ 

The Secondary Developmental Reading (SDR) Program began in the* 
fcolumbus Public Sohoola in the fall of 1971 as a oomponent of the Ohio 
Disadvantaged* Pupil Program fund. The 1982-83 version of the SDR 
Program was located in eight Columbus senior high school buildings. 
Twelve project reading teachers worked in these eight schools with 843 
pupils in grades 9-12 who soored- at or be}ow the 36 th percentile on a 
standardized achievement test in reading. 

\ 

i 

Within the 1982-83 SDR Program five teaohers in three .senior high 
schools participated in .a pilot project which utilized Ccimmodore PET 
computers for computer assisted instruction and computer managed 
instruction (CAI/CMI). v For evaluation purposes, CAJ/CMI related' data 
were collected from only two of these sohools- The third school did not 
become a CAI/CMI pro Jeot sohocl until December, and evaluation data from 
this sohool were limited to Regular SDR program data. The oomputers, 
software, and attendant ^services were contracted with the Prescription 
Learning Laboratory Company' of Springfield, Illinois. ' In addition to 
providing a new technique tei reading and l^ngu^ge instruction, the use 
of CAI/CMI was alsoi intended to enable teachers to serve more pupils 
than would be possible in regular SDR classrooms. The use of CAI/CMI 
was also intended to be a cost-effective alternative to replacing badly, 
worn conventional equipment. Of the 843 pupils in the SDR program, 261 
received computer assisted instruction. 

The purpose of the SDR Program is s to assist underachieving senior 
high pupils in raising their reading and communication skills. Emphasis 
oif the program is placed on literaoy survival skills necessary to 
function in, our word-oriented world. 

Features of the SDR Program include: 

1. Diagnostic testing to assess a pupil's individual 
reading, strength.? and weaknesses. 

Z. Individualized instruction tailored to meet the needs of pupils.. 

< 

3. Small group instruction. 

4. On-going evaluation of pupils to assess their reading nees. 

5. Inservioe meetings for teachers* 



Evalua tion Ob 1eo.tivt>§ 

"8 ' . * 

Objective 1i1 An evaluation sample wi\l «be comprised of pupils who' 
. score at; or below the 3&Sile on a pretest and are in attendance at > 
least BQ% of the instructional "period.. .The average reading growth of 
pupils *in the evaluation sample and participants in the Prescription 
Learning Laboratory will be 1.5 NCE points for each* month of 
instruction.. » 

- ^ > & 

Objective 1 , g Participants in the Prescription Learning Laboratory who 
have attended at least 80* of 'the instructional period will Bave passed 
an average of seven prescriptive readings skill objectives from the time 
of- the placement test to May 27, 1983 as vmeasured by the Prescription 
Learning Laboratory Mastery Test. ^* • . 

Object i ve 2» 1 To provide at least two inservioe sessions to program 
personnel such that' at least 80* of the insenvice participants will rate 
each session as valuable in providing information that will assist them 
in carrying out theii* program responsibilities. 

Objective ^i1 After completing the Presoription Learning Laboratory 
inservioe designed to instruot teachers on operating teaching machine's, 
instructing pupils^ in their use, prescribing instructional strategies, 
and maintaining a computerized instructional management system, all 
teachers will be able to respond correctly to 80 percent of the items 
included in a teacher training package instrument administered to 
teachers on or before Ootober 15, 1982. 

Objective, In May 1983, all teachers in the project will indicate 

that .the insorvice activities, provided by the Presoription Learning 
Laboratory Company during the 1982-83 school year were of value in 
assisting them to use the teaohing. machines , instructing pupils in their 
use, prescribing^ instructional strategies, and. to maintain a 
computerized instructional management system. > 

C 

t * 

r- Evaluation n«.^gn "s . 

' ) ' v 

The evaluation design for the SDH\?rogram called for the collection, 
of data in three areas. 

* 

1. Pupil Census Information 

■» ^^^^ 

The Pupil Censfus Form was developed for the purpose of 
^ collecting pupil demographic and participation data in the 

Secondary Developmental Reading Program. Projeot teaohers 
• maintained the Pupil Census Forms for all. .pupils throughout 

the school year and completed these forms at the end of the ^ 

program year or when the pupils left the program. 

Data collected on the Pupil Census Form were the nunber of days 
the pupil was enrolled in the program, the number of days the* 
pupil was in attendance,' and the average number of hours per 
week / 
1 the project teacher served the pupil. A copy of f Pupil Census 
Form can be found in the Appendix. s * ■ 
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Standardized Achievement Test information 



J 



The purpose of the administration of the standardized 
achievement test was to collect pretest-poattest achievement 
data on all SDR Program pupils to detert*ine if • * . 

Objeotive 1.1 was aonieveji. The standard achievement test used 
was the Metropolitan Achievement Test* Reading Comprehension, 
Form JS (MAT; 1978). Secondary Developmental- Reading n 
Program teaohers administered these tests on Ootober 4-8, v 
1982, and again on May 2-6, 1983. The form, subtest and test 
' levels of the 'Metropolitan Achievement Test used for each grade 
level are listed .below: 

* *■ * 

% 

* * 

J - Tftgt^ Laval - 

Grftde Form Suhtaafc / Prataat Poattaan 

9 JS • Reading Survey Advanced 1 , Advanoed \1 

10 t JS Reading Survey Advjanoed 1 • Advanced 2 

.11 JS Reading Survey Advanced 2 Advanoed 2 . 

12 JS ' Reading Survey Advanoed 2 Advanoed 2 

Mastery in the. prescription reading skill objectives was determined?, 
by hands-on 'testing at the oomputer terminals for pupils. served in 
the CAI/CMI units. A oazimum of £0 objectives oould be mastered .by 
high school CAI/CMI pupils. If a pupil 'mustered all of ^ these, he 
would then reoeive instruction in the same 20 basio objectives at. 
the next higher instructional level. The time of pretest varied • 
with the time a pupil began receiving oomputer assisted 
instruction. Posttest data consisted of all skills mastered by 
April 30, 1983. • , 

Inservice Evaluation Information ^ * • 

a. l>e teachers 1 ' perceptions of the value of tJ.o inservioe was to 
be determined by the rating of the Inservioe participants on the 
General Inservioe Evluation Form,. A copy of this Instrument 

' appears In the Appendix. The Inservioe programs were intended 
to assist project teaohers in teaching remedial secondary 
reading. A modified version of the General Inservioe Evalu- 
ation Form was used at the orientation meeting whioh occurred 
September 7, 1982. A oopy of the modified form also appears 
in the Appendix. \ 

b, 'The locslly developed Prescription Learning Laboratory In- 
servioe Assessment Form Was designed to assess the knowledge 
gained by CAI/CMI teaohers' from initial instructional meetings 
which were presented by the Prescription Learning Laboratory 
Company. The meetings for high* school CAI/CMI teaohers 

' occurred on September. 14-15, 1982. In addition to itisma of 
instructional content, (he instrument also contained rating 
scale items for teachers to rate the quality of the inservioe 
meetings and the quality of the services of the visitilg ' 
company consultants in helping them to implement the program. 
A copy of the instrument is found in the Appendix.* 



o. The CLEW and SDR Computer Training Evaluation Form waa de- 
signed to obtain ratings by CAI/CMI teachers of the usefulness 
of the, overall' CAZ/CMI inservioe activities for the 1982-83 
] school year.' The instrument was distributed in April, 1983, 

' atad collected in May, 1983. There Was a total of three CAI/ 

CMI inservioe meetings in the 1 982-83 school year. These 
occurred September "14-1 5, and March 22. The ratings also 
took into consideration the ongoing help provided by visiting 
company consultants, and printed materials, provided by the 
Prescription Learning Laboratory Company. A oopy of the in- 
strument ^s found in the Appendix. 

r * 

In addition to the types of data specified in the evaluation design, 
process evaluation data were obtained in a series of on-site visits to 
program classrooms. Observations, were ^onducteo* £33 all SDR units during 
the pretest administra^on of the standardized achievement test (October 
4-8, 1982), .and in ^ representative sample of SDR units. during the 
administration of the post test (Ma/ "2-6, .1983). These observations were 
conducted by personnel from the Department of State and Federal Programs 
and the Department of Evaluation Servioes. . The purpose of pretest and 
posttest observations was to obtala pertinent information regarding 
testing environment and test administration. Instruments used were the 
Chapter 1 and DPPF Pretest Observation Soale, tfnd the Chapter 1 and DPPF 
Testing Observation Soale. The latter: instrument waa used for posttest 
observations. Observations were also conducted during the- sohool year 
by a projeot evaluator to two high schools having CAI/CMI units, where 
four teaohers in the pilot project were interviewed. Data collected in 
the CAI/CMI observations included teaoher responses to an informal 
interview intrument, Questions for PLL Labs. A copy of each of the 
observation instruments is found in the Appendix. 



Malar HinHn T « 

Due to the faot that the 1982-83 SDR Program 'contained two treatment 
groups (regular instruction group and CAI/CMI group) data on 
enrollment /at tendance and achievement testing are reported below in two 
wayu.' These data are finst presented for the overall program regardless 
of treatment group. The seoond presentation oompares the two treatment 
groups in regard to enrollment/attendance data and achievement test 
data*-. * \ 

( 

In interpreting the preteat-posttest achievement data, the reader 
should be aware of the pupil selection process. Previous 
norm- referenced reading achievement data an€ staff reoojkendations were- 
used to .develop a pool of -pupils to be pretested. To/bw eligible for 
the program the pupil had to soore at or below .the 3*tb percentile on 
the pretest. Onoe the eligibility list was established, pupils were 
selected in order of their test scores with the lowest scoring pupils 
seleote'd first. 



During the 1982-83 school yf^r the SDR Program served 8*3 pupils. 
Of the' 843 pupils, 674 (80. Of) were ninth-graders, 122 (14. 5*) were 
tenth graders, 36 (4.3*) were eleventh graders, and 11 (1.3%) were 
twelfth graders. Of the 843 pupils, 422 (50.1*) attended ,tne minimum 
number of days (112) to meet the 80* attendance criterion level r ' 



contained in Objective 1.0. A breakdown by grade level showed that 356 
(52.8$) of the ninth-graders, 54 (44.3$) of the tenth-graders, 8 (22.2$) 
. of' the eleventh-graders, and 4 (36.4$) of the twelfth-graders met .the 
attendance criterion. The overall attendance rate for the program ;*as 
81.8$. The average number of days of enrollment and attendance for 
program pupils was '115.9 and 94.8 respectively.' The average' daily 
membership was 698.1. Table 1 contains, the pupil attendance data. 

The evaluation sample consisted of those pu^Lls who attended 80$ 
(112) of the 140 program days, and who ' received both a pretest and a 
o^ffttest in the Metropolitan Achievement Test. The sample also excluded 
^6 pupils, who were determined to bo non-finglish speaking. Of the 843 
pupils servecVby the program, 372 (44.1$) were in the evaluation sample. 

VThe analysis of pretest-post test achievement data provided minimums, 
maximums, averages, and differences for raw and derived scores by grade 
level. The derived scores used in the analysis Were percentiles, * grade 
equivalents, and normal curve equivalents. Caution should be exercised 
in interpreting the data at grades -11 and 12 because of the very small 
number of pupils who met the . attendance criterion and had both a pretest 

and a posttest score. 

* » 

.Raw scores are reported here for grades §, 11 and 12 only, since 
these are the only grades which reoeived the same level in both 
administrations of 'the test. The average 'raw score gain in ninth grade 
was'7.3 items which represented an increase* of 13.3$ of the 55 items on 
the Advanced' Mevel of the HAT. In eleventh/grade there was an average 
raw soore gain of 7.5 items, or 15.0$ of the ' 50 items on the Advanced' 2 
level on the MAT. In the twelfth grade there was an average' raw score 
gainSof 25.0 items, or 50.0$ on the Advanced 2 level of the MAT. Raw 
seore data are presented" in Table 2. In grade 10, pretest and posttest 
were administered at different test levels, with the result that any 
pre test-post test comparison of raw scores would be meaningless for this 
grade. However, the use of alternative level testing was Judged to 
provide a better match between pupil ability and test difficulty. ' A 
preliminary study at the , time of test selection indicated that the ' 
comparison of derived scores (percentiles, grade equivalents, and normal 
Qurve equivalents) across different levels of the MAT would provide ~ 
suitable reliability and validity in the assessment of pupil progress. 

* * * 

■ Table 3 contains pretest-post test percentile data. The median 
percentile for the pretest ranged from 8.0 at grade 12 to 19.8 at grade 
9. The median percentile for the posttest ranged from 18.5 at grade .11 
to 89.0 at grade 12. These data indicate that * while there was 
improvement at all grades, only at twelfth grade was the median . 
posttest percentile score at or above the 36th percentile. 



Table 1 



Number of Pupils Served, Averages for Days of Enrollment, Days of Attendance, 
Dally Membership and Hours of Instruction Per Week, and 
Pupils Attending 80$ of Days 
Reported by Grade Level 



/ 





.' 


• 




• 


• 

1 


Average 




• 

Pup'il3 /t 

Attending 
80$ of Days 


Grade 


Pupils 
Served 


Girls 


Boys 


Days of 
Enrollment 


Days of 
Attendance 


Daily ' 
v Membership 


■ ■ — «- 

Hrs. of Inst. 

' Per Pupil Per Week 


9 


, 674 * 


303 


' 371 


. 120.0 


t 

98.2 


57^.9 


' 4 

3.6 


356 


10 


122 


*42 


80 


102.4 - 


85.0 . 


89.3 


3.6 


54 \ 


11 

t 

12 


36 
.11 


19 
4 


7 


92,8 
90.6 


. 72.9 
69.6 


23.9 
7.1 


3.6 
3.6 

% 


•4 

* 


Total 

— « r 


843 


366 


' 475 


115.9 


■ 94.8 


A 

698.1 * 

0 


3.6 


422 
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Table 2 
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All Secondary Developmental Reading 
Minimum, Maximum, Average and Standard Deviation 
of the Pretest and Post treat Raw Scores 
, Reported by Grade Level ° 



• 

Grade 


Number 
3 Of Test 
Items 


Number 
of Pupils 






• 

Pretest 






r- 


Post test 






. Mln. 


Max. 


Average 
' Raw 
Score 


Standard 
Deviation 


Mln. 


Max. 


Average 

Raw 

Score 


Standard 
Deflation 


Average 
Change 


9 

♦ 


55 


322 


5 


35 


26.0 


7.0 




55 


33.3 


9.6 


7.3 

• 


11. 


50 


4 


7 

** 


29 


v 19.3 


11.4 


14 

« 


37 


26.8 


9.8 


' 7.5 


12 


, -50 


1 


23 


. 23 


23.0 


o.b 


*i|8 


• 48 


48 • 


0.0 


25.0 



Note. Raw scores are not reported for grade 10 because that grade received different 
levels of the test In pretest and post test administrations. A comparison of 
raw soores across different test levels would be meaningless, since item oontent 
and number of items varied across the two test 'levels. * 
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fable 3 



Minimum! Maximum, Median, and Standard Deviation 
of the Pretest and. Posttest Percentiles 
Reported by Grade Level 



Grade 


Number 
of Pupils 


Min. 


Max. 

ft 


Median 
Percentile 

~a 


Standard 
Deviation 


Mi*. , 


Max. 


Median 
Peroentile 


Standard 
Deviation 


9 


322 


1.0 


36.0 


19.8 


11.4 


1.0 


99.0 


27.6 


22.3 


10 


' 45 


1.0 


36.0 


10. 4 


9.4 


1.0 


98.0 


22.8 ' 


19.5' 


1 11 


4 


1.0 


24.0 


■ 12.5 


. 13.3 


1.0 


36.0 


18.5 


14.8 


'•' 12 


1 


8.0 


8.0 


8.0 




89.0 


89.0 


89.0 




, Total 


372 

% ■ 






18.11 


11.3 






26.3 


22.1 
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Table 4 contains pretest-posttest grade equivalent data. The ^median 
grade equivalent for the pretest ranged from 2.7. at grade 11 to 6.7 at 
grade 12. The median grade equivalent for the post test ranged from 7,4 
at grade 9 to 12.9 -at grade 12. These data indicated that: (a) there 
was a considerable inorease in the median grade equivalent at grades 11 
and 12, -(b) there was a greater than expected Increase in the median 
grade equivalent at all grades, and (c) that despite the gains', the 
pupils at the end of their ninth, tenth and eleventh grade years had a 
median grade equivalent at least two years below grade level in reading 
achievement. 

The presentation of achievement data thus far has included results 
from- the analysis of raw scores', percentiles, and grade equivalent*. 
Raw scores are equal units of measurement, but can only provide' a 
limited interpretation of achievement data. Percentiles and grade 
equivalents provide comparative information but are not equal units of 
measure. Caution is advised in drawing conclusions about program impaot 
from any, of the scores above. Normal curve equivalents (NCE's) are 
generally, considered to provide the truest indication of student growth 
in. achievement, since they provide comparative information in equal 
units of measurement. Data for nqrmal curve equivalents are presented 
in Table 5. 

i 

Objective 1 . 1 states that the evaluation sample would be composed of 
pupils who scored below the 36th percentile on the pretest and were in 
attendance 80* of the program's treatment period. In order to meet the 
attendance criterion the pupil had to attend at least 112 days of the 
seven month (140 days) -treatment • period. To achieve Objeotive 1.0 the 
average growth in reading achievement of pupils in the evaluation sample 
had to be 1.5 NCE's for each month of the treatment period which is an 
average of 10.5 NCE's for the seven month program. 

The overall NCE gain for the program was 7.6 or an average of 1.1 
NCE's for each of the seven months of the treatmeat period. This did 
not meet the evaluation criterion of 1.5 NCE's gained for every month 
the pupils were in the program. The greatest gain , shown at the 
twelth grade level where a total NCE gain^of 55.4 was recorded . (an 
average of 7.9 NCE's/month). However, caution should be exercised in 
any Judgement of NCE gains for grade 12 because the sample consisted of 
only one pupil. 

It should be noted that NCE scores are based on percentiles, whioh 
compare the pupil's performance in relation to the general population. 
No change in NCE score would indicate that a pupil has progressed 
normally over the school year. Even a small "gain in percentile or NCE 
score would indicate that a pupil has advanced over the school year at a 
greater rate than would be expected from the pupil's original position 
in relation to the general population. This becomes evident when we 
note that substantial progress was made at all grade levels in regard, to 
grade equivalent score*. Table 6 contains data related to the changes 
in NCE scores for three ranges: (&) decrease in NCE scores, (b) no 
change in NCE scores, and (c) gain in NCE scores. The data indicates 



Table H 

Minimum, Maximum, Median and Standard Deviation 
of. the Pretest and Posttest Grade Equivalents 
Reported by Grade Level 











Pretest 








Posttest 


1 % . ■ • 1 III! 


Grade 


Number 
of Pupils 


Min. 


Max. 


Median 
Grade 
Equivalents 


Standard 
Deviation 


Min. 


Max. 


Median 
Grade 
Equivalent 

• 


Standard 
Deviation 


.9 


322 

* 


1.5 


7.7 


5.8 


1.7 


1.8 


12.9 


7.4 


2.6 


10 


45 


2.2 - 


8.8 


5.7 


1.7 


2.3 


12.9 


8.3 


2.H 


11 


H 


1.8 


8.5 


2.7 


3.7 


2.8 


10.5 


8.2 


3.3 


. 12 


1 


6.7 


6.7 


6.7 




12.9 


12.9 


12.9 




Total 


■372 






• 

5.8 


£ 1.7 






7.1 


2.6 
















N 




^ 1 1, 



9 

ERIC 



4 



Table 5 



Minimum, Maximum, Average, and Standard Deviation of the 
Pretest and Poatteet Normal Curve Equivalents (NCE) 
Reported by Grade Level 



Poatteafc 



f 

Trade 


Number 
of Pupils 


Min. 


Max. 


Average 
• NCE 


Standard 
Deviation 


Min. 


Max. 


Average 
NCE 


Standard - 
Deviation 


Average 
Change « 




322 


i.o( 




29.7 


11.3 


1.0 


99.0 


36.9 


16.8 


7.2 


10 


45 


1.0 


42.5 


25.5 


10.0 


1.0 


93.8 


3^.0 


15.3^ 


9.5 


11 




1.0 


35.1 


18.1 


19.7 


1.0 


42.5 


26.. 3 


18.0 


8.2 


12 


1 


2D.it 

• 


20.4 


20.4 




75.8 


75.8 


75.8 




55.4 


•Total , 


372 


* 

r 


/ ' 


29.1 


11.3 






36.7 


16.7 


7.6 



1/ 
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that 250 (67.2*) pupila made gains in NCE' scores. The fact that 250 
pupils did better than expected indicates -that these pupils who met the 
attendance criterion benefitted from • the- individualized instruction' 
provided by the program. The least gain in NCE' a occurred at grade nine 
where the total NCE s gain was 7.2 or' 1.0 NCE 'a/month. • 

'Tabled , 

Changes J,n NCE Scores 

for All Grades * 







Number 

of Pupila 




Decrease in NCE Scores 


-29.8 to -0^.7' 

• 

»» 

' 0 


97 


23.4 


No^ Change in NCE Scores 


3$ ' 


9.4 


Gain in NCE Scores 


' * 

0.7 to 70.3 

» 


250 * 


67.2 



1 



Tables 7-11 present comparisons , between the pilot group of pupils 
receiving computer assisted/computer managed instruction (CAI/CMI) in 
reading and the group receiving the regular program instruction. As 
indicated in Table 7, there ^were 261 pupils .served by the pilot project 
and 582 pupila who received regular reading instruction. The CAJ/CMI 
group averaged more days of attendance, per pupil with an overall average 
of 99.5 days as compared to 92.7 days for the regular group. The 
Regular group, however, did average more days of* attendance at the tenth 
and eleventh grade levels. The evaluation sample of 372 pupila was 
comprised of 154 pupils in the pilot group and 218 pupils in the regular 
group. 

* 

Achievement data fo^ the two subpopulations of the program are 
presented in Tables 8-1 1 . It will be noted, that the number of eleventh 
and twelfth grade pupils in the evaluation sample is very small for both 
groups, and caution is advised in interpreting data for those grades. 
There were no twelfth grade pupila in the regular group and only one 
pupil in the CAI/CMI group. The number of tenth grade pupila is also 
relatively small in both groups. ' A greater degree of validity, can be 
assumed in comparisons for grade 9, where there were 134 pupils in the 
pilot project and 188 pupils , in the regular group. It is also assumed 
that valid group comparisons across grade levels can be made. 

Table 8 indicates greater improvement in raw score for the CAI/CMI 
group than for the regular group. The average change scores were 8.1 
items for the CAI/CMI group and 6.7 items for thjf regular group in ninth 
grade, and in the eleventh grade the average change scores were 9.3 
items for the pilot group and 2.0 items for the regular group. An 
impressive change score of 25.0 items ooourred in tho CAI/CMI twelfth 
grade, where there was only one pupil. There were no twelfth grade 
pupils in the regular group. Raw soore data are not reported for grade 
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Table 7 



Number of Pupils Served, Averages for Days of Enrollment, Days of Attendance, 
Dally Membership and Hours of Instruction Per Week, and. 
Pupils Attending B0% of Days Reported by Grade Level 
for Pupils Reoeiving Reading Instruction with Computers (CAI/CMI Group) 
and Pupils Reoeiving Reading Instruction without Computers (Regular Group) 



PupJJ.. 

Grade Served 


Girls 


Boys 




Average 




Pupils 
Attending 
bOf of Days , 


Days of 
Enrollment 


Days 'of 
Attendance 

* 


Daily 
Membership 


Hrs". of Inst. 
Per PudII Per Week 


tyl/CMI Group 




r 
















9 


205 


94^ 


111 


122.6 


106.8 


t79.5 


3.5 




139' 


10 


43' 


17 


26 


94.2 


75.4 


28.9 


3.5 




17 


11 


11 


7 


4 


73.6 


62.6 


' 5.8 


3.5 




3 


12 


2 


0 


2 


89.5 


71.5 


1.1 


3.5 




1 


Total 


261 


118 


143 


115.6 


99.5 


215.4 


3.5 




160 


RfigularvprouB 














♦ 






9 


469 


209 


260 


118.9 


94.4 


398.5 


3.6 




217 


10 


79 


25 


54 


106.9 


90.2 


60.3 


3.6 


37 . 


11 


25 


12 


13 


101.2 


77.4 


18.1 


3.6 




5 


12 


.9 






90.9 


60. 2 


S.fl ' 


1.6 




3 


Total 


582 


250 


332 


116.1 


92.7 


482.7 


3.6 




262 



9 



2u 



2i 



CO 



Table 3 



Minimum, Maximum, Average, /and Standard Deviation 
of the Pretest and Posttest Raw Soorea Reported by Grade Level 
for Pupils. Receiving Reading Instrudtion with Computers (CAI/CMI Group) 
and Pupils Receiving* Reading Instruction without Computers (Regular Group) 



Pretest 



Number 
Of ,Test 
Grade Items * 



Posttest 



12' 



50 



Number 
of Pupils „ Mln. 



Max. 



Average 

Raw Standard 
Score Deviation 



Mln. 



Average 
* Raw 
Max. Score 



Standard 
Deviation 



0 



Average 
Change 



CAI/CMI Group 
















* 1 






9 55 


134 


6 


35 


27.9 * 


7.1 


9 


54 

.5- 


36.1 


9.8 


. 8.1 


11 50 


3 


7 


29 


16.0 


11.5 


14 


37 


"25.3 


11.5 


9.3 


. 12 50 


1 


23 


23 


23.0 


0.0 


48 


48 


48.0 


0.0 


25.0 


Regular Group 








% 


f 












9 . 55 . 


168 


5 t 


35 


21.7 


6.6 


* 13 


55 


31.4 


9.P 


'6.7 


11 50 


1 


29 


29 


2970 


0.0 


31 

§ 


31 


31.0 


0.0 . 


2.0 



Note* Raw scores are not reported for grade 10 because that grade received different 
levels of the test in pretest and posttest administrations* A ooaparison of 
raw scores across different test levels would be meaningless, since item content 
and number of items varied aoross the two test levels. 
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Table 9 



Minimum, Maximum, Median, and Standard Deviation 
of the Pretest and' Poattest Percentiles Reported by GrHe Level 
for Pupils Receiving Reading Instruction with Computers (CA1/CMI Group) 
•and Pupils Reoeivlng Reading Instruction without Computers (Regular Group) 



♦ 




















• 


frGmber 
of Pupils 






Pre teat 








Post test 


1 


Grader 

9 


Min. 

9 


Max. 


Median 
Percentile 


Standard 
Deviation 


Min* } 


Max. 


Median ' . 
Percentile 


• 

Standard ^ 
Deviation *- 


CAI/CHI 


Group 








r 











9 


134 


1.0 

« 


36.0 


26.3 


11.6 


1.0 


98.0 


35.8 


23.7 


10 


16 


1.0 


28.0 


15.0 


8.0 


1.0 


98.0 


20.5 


29.1 


11 




1.0 


24.0 


6.7 


)13.3 


1.0 


36.0 


14,0 


17.7 


12 


1 


8.0 


8.0 

^— . . , - 


8.0 


, 0.0 - 


89.0 


89.0 


89.0 


0.0 




r 


















Total 


151 • 

I 






24.0 


11.6 




• 


32.5 


24.5 


Refular 


-Oram* 1 








* 


- 


• 






9 


188 


1.0 


36.0 


16.2 


10.5 


1.0 


99.0 


23.7 . 


20.0 


10 


29 


( 2.d 


36.0 


9.5 


10.1 


4.0 


46.0 


23.0 


10.6 


11 

» 


1. 


24.0 


24.0 


24.0 


0.0 


23.0 " 


23.0 


23.0 


0.0 


12 


' ' 0 * * 


















Total . 


- 218 






15.6 


10.5 

S,. 






23.6 


18.9 



Table 10 

a 

Minimum, Maximum, Median, and Standard Deviation 
of the Pretest and Postteat Grade Equivalents Reported by Grade Level 
for Pupils Receiving Reading Instruction with Computers (CAI/CMI Group) 
and Pupils Receiving Reading Instruction without Computers (Regular Group) 















/ 








• 


• 

Grade 


' Number 
of Pupils 


Min. 


■ 

Max. 


Median 

VI 1 QUC 

Equivalents 


V — 

5?t ft nrlfl nri 

Deviation 


** 

. Min. 


Max. 


Median 

VI I QUO 

Equivalent 


AjWoiiuai va 

Deviation 


CAI/CMI 


Ajroup 








• 




— 




s 




9 r 




131 


1.6 


7.7 


6.7 


1.7 ' 


•1.8 


12.9 


8.3 


2.7 


10 




16 


2.2 


8.0 


6.2 


1.5 


2.3 


12.9 


7.9 




11 
1 I 




3 


1 fl 






Q 7 


9 A 


1 n k 


7 il 


Q Q 


1 0 




1 


fi 7 


fi 7 




U . U 






* 

IO Q 




Ttfttel 




154 






6.6 


1.7 






8.3 


2.8 


Regular 

9 




168 


1.5 


7.7 


5.3 


• 

1.6 


2.4 


12.9 




2.4 


10 




29 


2.8 


8.8 


5.3 


1.8 


3.8 ^ 


10.5 


8.3 


1.7 . 


11 

12 




1 

0 


8.5 


'8.5 


* 3.5' 


0.0 


8.9 


8.9 


fl - 9 «- 


0.0 


Total 




218 






5.3 


1.6 






7-0 


2.1 



Table 11 



•Minimum, Maximum, Average, and Standard Deviation of the 
Pretest and Posttes^ Normal Curve Equivalents (NCE) 
v Reported by Grade Level 

for Pupils Receiving Reding Instruction with Computers (CAI/CMI Group) 
and Pupils Reoeiving Reading Instruction without Computers (Regular Group) 



t 




0 

k.. ... . 




Pre teat 


• 






Posttesfc 






Number 
Grade' of Pupils 


Min. 


Max. 


Average 
NCE 


Standard' 
Deviation 


Min. 


Max. 


Average 
NCE 


Standard 
Deviation 


Average 
* Change 


CA1VCMI Group 






















9 


4 

134 


1.0 


42.5 


32.5 


1 1.2 


1 .0 


• j 


41 .7 


17 3 


Q 2 


10 


16 


1.0 


37.7 


27.5 


9.2 


1 .0 


•* j • j 


37.8 




10 3 


11 . 


3 


1.0 


35.1 


12.4 


19.7 


1 .0 


42.5 

~V»- • 


23.6 


21 0 


IIP 


12 


1 


20.4 


20.4 


20.4 


0.0 


75.8 


75.8 


75.8 


0.0 




Total 


154 






31.5 


11.6 






41.2 


18.3 


9.7 


Regular Group 






















9 


188 


1.0 


42.5 


27.8 


10.9 


1.0 


99.0 


33.6 


15.6 


5.8 


10 


29 


6.7 


42.5 


24.4 


10;4 


13.1 


47.9 


33.5 


8.5 


9.1 


11 


1 


35.1 


35.1 


35.1 


0.0 


34.4 


34.4 


34.4' 


0.0 


-0.7 


12 


0 












* 








Total 

o — . 


218 






* m — - 

27.4 


10.8 






33.6 


U.8 


6.2 



ERIC 



\ 



2d 



2H 



10 due to the fact that tenth grade pretest and post teat were given with 
different levels of the test, which precludes comparison of raw score 
data at ttfat grade. 

Good progress in terms of median percentile soores was indicated for 
both groups, as seen in Table 9. The median percentile across grade 
levels increased from 24.0 to 32.5 in the CAI/CMI group, and from 15.6 
to 23.6 in the Regular group. Similar gains are seen in terms of median 
grade equivalent scores, as presented, in Table 10. ' The median grade 
'equivalent score across grade levels increased from 6.6 to 8.3 in the 
CAI/CMI group, and from 5.3 to 7.0 in the Regular group. i 

As indicated earlier, NCE scores are generally considered to provide 
the most comparative information in equal units of measurement. Data 
for the two' groups in terms of NCE scores are presented in Table 1 1 . 
The data indioate that greater average NCE gains occurred for the 
CAI/CMI group than for the Regular group at all grade levels. The 
average NCE gain across grade levels was 9.7 for the CAI/CMI group, and 
6.2 for the Regular group. Grades 10, 11, and 12 of the CAI/CMI group 
achieved the evaluation oriterion of 1.5 NCE'a gained for every month in 
the" program, with average monthly gains of 1.5, 1.6, and 7.9, 
respectively. The gains at grades 10-12 should be interpreted 
cautiously, however, due to the small number of pupils at those .grades. 
When average gains are considered across grade levels it is evident 
that, although neither the CAI/CMI group nor the Regular group achieved 
the evaluation criterion . of an average NCE gain of 1 .5 per afenth of 
instruction, greater progress was achieved in the CAI/CMI group. 

Objeotive 1.2 speoified that pupils in the CAI/CMI project who 
attended at least 80? of the instructional period would make an average 
gain of seven prescriptive reading skill objectives as measured by the 
Prescription Learning Laboratory Mastery Test. Testing occurred at the 
oomputer terminals using software supplied by the Prescription Learning 
Laboratory Company. Averages and standard deviations for the 
prescriptive reading skill objectives are presented in Table 12. The 
following average gains were made in the number of skill objeotives 
mastered between pretest and post test: ninth grade, 8.8 objeotives; 
tenth grade, 9.2 objeotives; eleventh grade, 4.7 objectives; twelfth 
grade, 10.0 objeotives; and an overall average gain of 8.8 skill 
objectives. Objeotive 1.2 was attained by the project. It was also 
attained at each grade level except grade eleven, where there were only 
three pupils in the . sample. One caution is advised in interpreting 
mastery test data: the pretest was a placement test, which did not cover 
all skill objectives in the ooatinuum. Therefore it is not certain that 
gain scores can be attributed entirely to treatment oocurring between 
pretest and ppstteat. However, there was a small positive correlation 
of .27 between gains in the Prescription Learning Laboratory Mastery 
Test and NCE gains on the Metropolitan Achievement , Test, as calculated 
by the Pearson produot-moment formula. 
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Table 12 

Average Sobres and Standard Deviations of the Pre teat, Poattest, 
and Change in Skills Mastered in the Freaoription 
Learning Laboratory Mastery Test by Pupils in the 

CAI/CMI Project 

4 • 



Grade 



Pretiat Skills M««fc«n«H 

Number Standard 
of Pupils Average Deviation 



Poattaat Skills H*atm»mA 

Standard 
Average Deviation 



Change. 



Average 



Standard 
Deviatio 



9 


134 


5.6 


2.6 


14.4 


5.9 


8.8 


5.6 


10 


16 


5.2 


2.6 


. 14.4 _ 


5*1 .... . 


9*2 





11 


3 


3.3 


. 3.1 


8.0 ' 


7.0 


4.7 


4.0 

• 


12 


1 


10.0 


0.0 


• 10.0 


0.0 


10.0 


0.0 


Total 


154 . 


5.5 


2.7 


14.3 * 


•5.9 


8.8 


5.5 



Objective 2.1 stated that program personnel would be provided at least two 
inservioe sessions and that at least 80$ of the personnel attending each session 
would rate the session as valuable in providing information that would assist 
them in carrying out their program responsibilities. Because of budgetary 
limitations, these specified inservioe sessions were . not held. An orientation 
program was held on September 7, 1982 and an evaluation was conduoted (see Table 
13). It should be noted, however, that even this session was optional slnoe not 
all teaohers could be released from their duties to attend. Of the six teachers 
who attended the session, five agreed or strongly agreed that the information 
provided would help them in their program. 

Although only one inservioe meeting was available for the overall group of 
SDR teachers, there were an additional three inservioe meetings for those SDR 
teachers who participated in the CAI/CMI pilot project. Orientation meetings for 
instruction in using the computers and Implementing the project were held 
September 14 and 15, 1982. A follow-up meeting was held March 22, 1983. 
Objectives 3*1 and 3*2 pertain to Inservioe for the CAI/CMI teaohers. 

Objective 3*1 required that all CAI/CMI teaohers be able to respond oorrectly 
to at least 80 percent of the items on an instrument dealing with content of the 
inservioe sessions. The Prescription Learning Laboratory Inservioe Assessment 
Form was designed to provide evaluation data on this objective, and also to 
provide an opportunity for teachers to rate the inservioe provided. The 
instrument was administered following the two initial inservioe meetings. 
Results obtaine'd ~>£rom the questions on inservioe content indicated that the 
objective was not attained, with all four participating teachers scoring below 80 
percent. 

31 



20 



In rating the first two 'inaervice meetings, high school CAI/CMI teachers 
assigned a median rating of 3 . (good) , on a five-point scale to inaervice 
instruction on instructing pupils in the use of teaching machines. However,, 
three other criteria on which the instructions were rated received median ratings 
placing between poor and satisfactory. These criteria and their ratings weres 
operating teaching machines 1.3, prescribing instructional strategies 1.5, and 
maintaining a computerized instructional management system 1.5. Teacher comments 
suggested an underlying, problem: $hat a great deal of material had to be covered 
in too short a time (two days). 

Table 13 

Average Responses and Response Frequencies 
for, September 7, 1982 



Rganonaas 



Number Average SD D D A SA 
Responding Response (1) (2). (3) (4) (5) 



General M««fcinq 

1 . I think the presentation by 
Dr. Michael Mil one was very 
worthwhile. 

2. The information presented 
by Dr. Michael Mil one will 
assist me in my program. 1 

Mini Sessions 

3. The exhibit of materials 
was very valuable. 

4. The information presented by 
Dr. Mil one during the mini- 
session will assist me in 
my program. 

5. The Chapter 1 ' mini-session 
heightened my awareness of 
overall program procedures. 



3.3 



3.6 



3.4 



3.2 



2.5 



6. The evaluation presentation 
will assist me to successfully 
complete this year's 
evaluation requirements. 



4 



3.3 
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Overall 

7. There was cime to ask questions 
pertaining to the presentations. 6 

8. Questions were answered adequately. 6 



9. The orientation meeting was 
worthwhile. 



i 



4* 



v 3.2 
3.5 

3.7 



1- 
0 



0 
0 



4 
5 



0 
0 



The Prescription Learning Laboratory Inservice Assessment Form also- 
obtained teachers' ratings of services provided by the visiting 
consultants from Prescription Learning Laboratory Company. ""the 
consultants visited each high school CAI/CMI unit twice, a month, 
providing ongoing inservioe and help with special problems. High school 
CAI/CMI teachers rated the aervioes of the consultants on the same 
criteria that were used to rate the inservice meetings. The following 
median ratings were obtained on a five-point scale: operating teaching 
machines 2.0 ( satisfactory )y instructing pupils in the use of teaching 
maohines 2.0, prescribing instructional strategies r.O (poor), and 
maintaining, a computerized instructional management system 1.5. 

Objective 3.2 stated "all teachers in the project wil indicate that 
the inservioe activities provided . by the. Prescriptive Learning 
Laboratory Company. . .were of value in assisting them to use the teaching 
maohines, instructing pupils in their use, prescribing instructional 
strategies, and to maintain .a computerized instructional management 
system." The CLEAR and SDR Computer Training Eval^jation Form was 
designed to evaluate this objective. The instrument used a five-point 
scale ranging from "strongly disagree" to "strongly agree," with . a 
mid-point of "undecided". Data were collected in toy from the three 
remaining teachers who had started in the project at* the beginning of 
the school year. No CAI/CMI data were oolleoted from two teaohers in a 
school where a CAI/CMI unit had been installed in Deoember. There had 
been an additional inservioe meeting in March, and the timing of the 
data oolleotion near the end of the school year permitted an overall 
perespective of the inservice activities provided in the first year of 
the pilot project. For purposes of evaluation, inservioe activities 
were defined to include the three^training sessions, the ongoing help of 
visiting consultants, and printed "* materials provided by the servicing 
company. Summative data from this instrument are presented in Table 14. 

In assessing the overall value of inservioe activities in regard to 
the four criteria stated in the objective, all three teaohers gave a 
rating of 4 (agree) on a five-point soale to eaoh of the four criteria. 
This would indicate, that Objeotive 3.2 was attained in regard to its 
separate Criteria.. Results were somewhat olouded, however, by teaohers' 
ratings of the value of inservioe activities in implementing the overall 
program: one teacher rated this item 4 (agree), but two teaohers gave a 
rating of 3 (undecided). Comments by teachers indicated that initial 
servioes of the visiting consultants during the year. was very helpful, 
and that more in-depth knowledge in utilizing computers was desired. 
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Table 14 



Averages and Frequencies of Teacher Ratings of the Value of Overall CAI/CMI 
Inservice Training to Areas of the Inservice Objective 



Area of Objective 



Rating 

1 2 3 4 5 

Strongly Strongly 
Rating Disagree Disagree Undecided Agree Agree 



Learning to use 

instructional 

machines 

Instructing pupils 
in using machines 

Prescribing 

instructional 

strategies 

Maintaining 
% computerized 
instructional • 
management system 

Implementing 

overall 

program 



3 
3 



4.0 0*0 0 

4.0 0 0 0 

* 

4.0 0 0 0 



4.0 



3.3 „ 0 



3 * 



p 

In addition to the types of data specified in the evaluation design, 
process evaluation data were obtained by means of on-site visits. Observations 
were made during the pretest and post teat administrations of the achievement 
teat, in order to gain first-hand information in regard to testing environment 
and test administration. Visits were also made during the year to the CAI/CMI 
pilot project. * 



Elements of the testing environment were generally judged to be good or 
very good. As peats of the testing environment that were checked inoluded 
lighting in the testing area, apace for each student, ground or noise level, 
and temperature. In the one case where the testing environment was judged to 
be less than acceptable, the problems are a result of the teacher having to use. 
a room jaexti>to the ventilation fans that produce excessive noise. 

The presentation of the test directions was r&ted as good or very good in * 
all cases. All teachers read the test directions and a few teachers 
demonstrated on the board an example of the method for marking the answers. 1 
All teachers followed the directions in the examiner's manual. One teacher was 
rated as "poor" in accuracy for timing the test. 

% ■ 
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During the testing sessions the appropriate materials were generally judged 
to be available and used in most cases* The most common, omissions were: 
failing to place a "Testing - Do Not Disturb" sign on the door (four cases), 
failing to provide pencils with eraserj for each student (four cases), failing 
to use a stopwatch, watch or clock with a second hand for timing the test (two 
oases), and failing to use a copy of the test booklet for demonstration 
purposes (two cases). f 

Several visits were conducted during the school year to the CAI/CMI pilot 
project units by a project evaluator. Interviews 1 with project teaohers in 
October and November indicated several problems in the implementation of the 
program. At that time teohnical difficulties with the oompufcera, and tapes had 
not been overcome. There was also a feeling among project teachers that 
instructional materials provided at the high school level were not suitable, 
and that diagnostic and prescriptive materials produced results which were 
suspeot. There were also delays in reoeiving some of the materials needed for 
instruction. On the positive side, the computers were considered to provide 
excellent pupil motivation. 

The program evaluation included one further analysis not in the original 
evaluation design: a cost-benefit analysis comparing the CAI/CMI group and the 
regular group. This analysis is summarized in Table 15. Costs inoluded in the 
analysis Included teacher salaries and the oontract cost for Prescription 
Learning Laboratory Reading Labs. Normal supplies and incidental costs were 
not known in regard to the two groups, but were assumed to be evenly 
uistrlbuted. Any error of cost estimate resulting from unknown costs would 
probably be in the direotion of underestimating the cost for the Regular group, 
since most instructional materials for the CAI/CMI group were included in the 
Prescription Learning Laboratory contract costs. The bost-benefit analysis 
indicated the cost per pupil based on average dally membership was 
approximately the same for both groups ($532.24 for CAI/CMI group and $529.80 
for Regular group). However, the CAI/CMI group's average NCE gain was 3.5 NCE 
points higher than for the Regular group. The CAI/CMI group also 'had better 
attendance, with $1.3? of the pupils meeting the attendance criterion and 59. 0% 
meeting all criteria for inclusion in the evaluation sample (attendance, 
pretest t and posttest, and English-speaking). In the Regular group, 45.0$ of 
the pupils met the attendance criterion and 37. 5% met all criteria for 
inclusion in the evaluation sample. 

Summary /V mGam mmndlLt±ana 

The Secondary Developmental Reading Program is an individualized learning 
program designed to assist secondary pupils who are having reading problems,. 
During the 1982-83 school year, 12 project teaohers working in eight senior 
high schools served a total of 843 pupils. 

The program had five objectives. Two objectives pertained to the program 
as a whole, and'* an additional thr^o objectives dealt with a computer 
assisted/computer managed instruction (CAI/CMI) pilot project which was a 
subset of the program. Objective 1.1 stated that pupils who attended 80$ of 
the seven month treatment period would show an average gain in reding of 1.5 
NCE's for each month which is an average gain of 10.5 NCE's overall (seven 
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Table 15 

Cost-Benefit Analysis for 1982-83 Secondary. Developmental Reading Program 
Comparing Group Reoelvlng Computer Assisted Instruction/Computer 
Managed Instruction (CAI/CMI) and Group Reoelvlng Regular Program Instruction 



Program 



Number 

of 
Teachers 



Prcferaa Coat 



Total 



Per 
Teacher 



Average Daily Mem bership Cost 
In Per ' Per 

Program Teaoher Pupil 



Pupils V 
Meeting 
Attendance 
Criterion 



Ratio of 
Sample 

to 
Pupils 
Served 



Average 
NCE * 
Gain 



SDR-PLL 
(grades 9-12) 
with, CAI/CMI) 



114,645 38,215.00 



SDR 

grades 9-12 

(Regular group) 9 255,735 28, 415. DO 



215.4 



482.7 



71.8 



53.6 



532.24 



529.80 



61.3? 



45. OS 



59. OJt 



37.5* 



9.7 



6.2 



3o 



* 37 

r 




months x 1.5 NCE's). This objective was not achieved. The average gain was 
7.6 or 1.1 NCE's/month. The grade equivalent data indicated that the "average" 
ninth- and tenth-grader in the program ended the year reading at approximately 
the mid-seventh and mid-eighth grade levels respectively. 

f 

i 

Objeotive .2.1 stated that program personnel would be provided at least "two. 
inaervioe sessions and that at least 80* of the personnel attending each 
session would rate the session as valuable in providing information that would 
assist them in oarrying out their program responsibilities. Objective 2.1 was 
not aohieved. Beoause of budgetary limitations only one session was held. It 
was attended by six of the 12 progrm teachers. All but one of the teachers 
attending the session rated the session as being of assistance to them in their 
program. 

The CAI/CMI pilot projeot was located in three high schools. Since one of 
the high schools did not convert to CAI/CMI until December, evaluation data for 
that sohool were included in overall program evaluation, but not in' the sample 
for the pilot projeot. The oomputer assisted units in the remaining two 
project sohools served 154 pupils. Although neither the pilot project group 
nor the group reoeiving regular program . instruction attained the achievement 
criterion, greater gains were noted in the CAI/CMI group. The averge gain in 
NCE's in a 'seven month period was 9.7 for the pilot group, compared to 6.2 for 
the regular group. 

^ Objeotive 1.2 stated that pupils in the CAI/CMI projeot who attended 80% of 
the seven month treatment period would gain an average of seven prescriptive 
reading skill objectives In mastery tests performed at the oomputer terminals. 
This objeotive was aohieved, with, an average gain of 8.8 reading skill 
objectives. 

The remaining two objectives pertali to inservioe activities provided by 
Prescription Learning Laboratory Company to teachers in the CAI/CMI projeot. 
Inservioe activities were intended to provide teachers with instruction in the 
following areas: operating the project's teaching machines, instructing pupils 
in the use of teaohing maohlnes, prescribing instructional strategies, and 
maintaining a computerized instructional management system. - These four/ areas 
of instruction furnished the criteria for objectives 3.1 and 3.2. 

Objeotive 3.1 stated that all CAI/CMI teachers would be able to respond 
correctly to 80 percent of the items, in an instrument dealing with content of 
the inservioe instruction. This objeotive was not attained. It appeared that 
the two days of inservioe instruction provided in September had not been 
sufficient considering the great deal of material that had to be covered. 

Objective 3.2 stated that all CAI/CMI teachers would indloate in May, 1983 
that inaervioe activities^ provided by Prescription Learning Laboratory Company 
during the school year were of value to them in the four areas of inservioe 
instruction identified above. In addition to the initial two day workshop in 
September, there had been a wrap-up inaervioe session in March. Inservioe 
activities were also considered to inolude, the on-going help provided by the 
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company* a visiting consultants , and printed materials furnished .by the 
company. This objeotive was attained , with all three teachers from the two 
schools in the CAI/CMI sample .agreeing that inservioe activities were helpful 
to them in each of the four areas of the objeotive (operating teaohing 
machines, instructing pupils in use of machines, prescribing instructional 
strategies, and maintaining a computerized instructional management system). 
The" teaohers registered a degree of uncertainty, however, to the value of the 
inservioe activities in, overall implementation of the project. 

A cost-benefit study indicated that the cost per pupil was approximately 
the same for the CAI/CMI group and the Regular group. However, the CAI/CMI 
group. produced' greater benefits in regard to NCE gains and pupil attendance. 

Turing the 1982-83 school year, the Secondary Developmental Reading Program 
continued to experience problems in several areas. 

1. Pupil attand&nea s Of the 843 pupils served by the program., only 
50.1 % attended 80* of the treatment period. The overall rate of . 
attendance was 81. S%, This is approximately k.0% below the 
district-wide average for senior high pupils. These data 
suggest that pupils are absent approximately one day per 

week of the program. 

2. Pupil achiftTemant t .While the reading achievement data in- 
dicated that pupils did improve their reading" skills, post- 
test reading levels were tar below grade level. 

Giyen the fa'ot that these problems have continued over several project 
years and given the drastic reduction in DPPF funding, it is recommended that 
an extensive review of evaluation results for all DPPF projects be conducted 
with an eye toward program' revision. 

In addition to general program conoerns, the following concerns expressed 
by teaohers in the new CAI/CMI pilot projeot should be addressed: 

1. More time should be allotted for initial inservioe instruction, 
in order to allow instruction in- greater depth than was possible 
in the two day September workshop. 

2. Diagnostic and instructional materials should be examined more 
closely for suitability to the high school level. 

3. Delays in receiving needed materials should be. eliminated as 
much as possible. 

Since the Secondary Developmental Reading Program is to be continued for 
the 1983-84 school year, consideration should be given to the following: 

1. Conduct the projeot in schools that will work with projeot 
personnel to reduce scheduling problems and increase program 
attendance. 
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Change the selection criteria so that the pupils are not 
selected' solely on the basis of being the lowest on the pre- 
test list. 1 Pupils and perhaps parents, should be asked to 
mate a commitment to attend the program classes. Pupil 
contracting should be considered in this regard. 

Review evaluation and program data to determine the match' 
between pupil characteristics, program objectives, and 
instructional activities, which will result in the most 
improvement in student performance. Evaluation procedures 
should be modified accordingly. 

Continue to evaluate the CAI/CMI pax*t of the program with an 
eye toward finding more effective methods of serving the high 
school pupil who is experiencing reading problems. 
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GENERAL INSERVICE EVALUATION FORM 



Inservice Topic: 
Presenter(s) : 



Date: ' . (e.g., 9/7/82) 

Session: a.m. or . p.m. 

(1) Chapter 1 (2) DPPF (3) General 
(circle only one) < 4 > dthe r < s P ec1f *) 



Program: (1) ADK (2) Aides (3) CLEAR-Elem (K-5) 

(circle only one) (4) CLEAR-M1ddle (5) HSCA (6) OND 

(7) SDR (8) Regular Teacher 

(9) Other (Specify) ' . 



Circle the number that indicates the extent to which you agree with statements 1-4. 

Strongly Strongly 

Disagree Disagree Undecided Agree Agree 

1. I think this was ,a very 

worthwhile meeting. 1*2 34 5 

2. The information presented in 

this meeting will assist me 1 2,34 5 

in my program. 

3. There was time to ask questions 1.2 3 4 5 
pertaining to the presentation. T - " 

4. Questions were .answered 12-34 5 
. adequately, 

5. What was the most valuable part of this meeting? ' • 



6. What was the least valuable part of this meeting? 



7. What additional information or topics would you like to see covered in future 
meetings? ; ( 
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ECrA CHAPTER 1 • 
ORIENTATION INSERVICE EVALUATION FORM 
September 7,, 1982 



Fund: 

(Circle only one ) 
^Program 

(Circle only one ) 



(1) Chapter 1 (2) DPPF - (3) General 
(4) Other (specify) 



(1) ADK (2) Aides (3) Chapter 1 - Elem. (K-5) 
(4) chapter 1 - Middle (5) HSCA (6) OND 



jjj ??? Regular Teacher 



Other (specify) 



Circle the number that Indicates the extent to which you agree with statements 1-9. 



Strongly 

Disagree Disagree Undecided Ag 



General -Meeting 

1. I tmnk the presentation by Dr. Michael 
MHone was verv worthwhile. 

2. The Informafon presented by Dr. 
Michael MHone will assist me In 
my program. 

Mini Sessions 

3. The exhibit of materials was very- 
valuable. 

4. The Information presented by Dr. MHone 
during the mini -session will assist me 
1n my. program. 

5. The Chapter 1 mini-session heightened my 
awareness of overall program procedures. 

6. The evaluation presentation will assist 
me to successfully complete this year's 
evaluation requirements. 

Overall * 

7. There was time to ask questions pertaining 
to the presentations. 

8. questions were answered adequately. 

9. The orientation meeting was wort».*hi1e. 



1 
1 



-1 
1 
1 



1Q. What was the most valuable part -of this meeting? 



2 
2 



2 
2 

0* 



3 
3 



3 
3 
3 



Strongly 
ee Agree 



5 
5 



5 
5 
S 



* 11. What was the least valuable part of this meeting? 



12. What additional information or topics would you like to see covered in future meetings? 
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PRESCRIPTION LEARNING LABORATORY 
• INSERVICE ASSESSMENT FORM . 

Part I. Inservic&i Rating Scale 

IIJ e thn r 5Mi^!I; L J nserv1 Sf worksh °P 1 ! September provided Instruction ** 

hC hLhII 0 !!" 9 area S Please rate the * u * m V of tnat Instruction 
by circling the appropriate numbered ratings. 

Poor Satisfactory Good Exceffent Superior 

a. Operating teaching machines" 1 2 3 4 ' 5* 

b. Instructing pupils 1n the 

use of teaching machines 1 2 34 5 

c. Prescribing Instructional 

strategies 1 2345 

d. Maintaining a computerized 

management system } 2 34 5 , 

Visits to your lab by the company consultant are meant to assist you to 
increase your proficiency 1n the following areas. Please rate the 
quality of that assistance by circling the appropriate numbered 
ratings. 

« 

Poor Satisfactory Good Excellent Superior 
a. Operating teaching machines 1 2 3 4 5 

p. Instructing pupils 1n the 
use of teaching 

machines 1 2345 

c. Prescribing Instructional 

strategies 1 2 3 4 5 

1 

d. Maintaining a computerized 

management system 1 2 3 4 5 



\ 



44 



- - PRESCRIPTION LEARNING LABORATORY 

INSERVICE ASSESSMENT FORM 7 

Part H. Content Assessmon f 

Please circle the one response that best answers each of th. , 

i uu - answers eacn of the following questions. 

1. When using the Callfbne. the student records response, 
• a. using the keyboard 

b. by pressing the "student" button 

1' ' P ms 3"9 t"e "record" button 

e! % naffi 0th -? ;" StUdMt " - "uttons 

2. The name of the filmstrip- projector used in the PLL lab is the 

a. Craig b. Hoff^ c. Oukane d. Audiotron e. Califone 

3. ™*ff*m on the PET cornier is used to add. delete, or update student . ' 
0 ' a. Hands on Testing b. Standardized Testlno c a*™*, m 

d ' . P «*ram e: e ^? n ^enanceXlr!S nU 

4. Student records are stored on the • 

a- nop* Oisk b. Tape e Ca S sette ois , Printer d. Oisplay Screen 

5> USfJWii! ^ Ma1n Henu *™ *" ' visual display of information 

a. Hands on Testing b. Standardized Testing e p.™,*. «. 

d. Inquiry Program e . tointenance l£g£ ^ 

6. The Lab Profile Report prints the skill continuum in regard to 
a. an Individual pupil 8 

h. pupils grouped by lab period 

c. pupils grouped by similar needs 

d. the report to parents 

e. the listing of available materials 

7. . «,. »„« „ ,„„„,. frumM to *. t «, ,„„ „ 

5. ngr^ studen<rshou '< U2t Press while listening to tapes In the ^ 
a. Rewind b. Forward c. Reverse d. Record e. Stop 
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Chapter 1 and DPPF Projects 
CLEAR AND SDH COMPUTER TRAINING EVALUATION FORM 

t 

Grade Level (Check one) Elementary__ Middle Schoo l High School _ 

This is an end«of-the year measure of how well certain inservice training 
activities have provided help for you in the following areas: 

t 

a. Using instructional machines 

b. Instructing pupils in the use of instructional maotines 

c. Prescribing instructional strategies 

d. Maintaining a computerised instructional 

management system 

For the purposes of this evaluation form, the term "inservice training" is 
broadly defined as follows: 

a. Workshops or training sessions presented by the company that 
supplies your computers 

b. Help from the visiting company consultants (elementary and 
* high school levels, only) 

o. lelp with technical difficulties via the toll-free telephone 
number (middle school level only) 

d. Instructions and explanations from printed materials supplied 
by the company - manuals, handbooks, program notebooks, etc; 

Circle the number vhiqfa indicates the extent to which you agree with the 
following statements (please give a rating to each sub-statement of all 
items that pertain to your level of instruction) : 

Strongly Strongly 

Diaa^raa Dlsagraa TTnrinfliHftd Agraa AgClS 

1. Workshops or training 
sessions by the company 
that supplies our computers 
have been valuable to me in 

>* 

a. learning to use the 

instructional machines ,1 2 3 4 5. 

b. instructing pupils in. 

using the instructional 1 v 2 • 3 4 5 

machines 

c. prescribing instructional 

strategies 12 3 4 5 

d. maintaining a computerized 
instructional management 

system 12 3 45 

e. implementing the' overall 

program 1 % • 3 4 5 

4b 



Strongly 

Blaigcu 



Strongly 



(To be completed by 
elementary and high school 
teachers only) 

The services of the visiting 
oompany consultants have 
been valuable to ae in 

a. learning to use the 
instructional machines 

b. instructing pupils in 
using the instructional 
machines 

* 

o. prescribing instruc- 
tional strategies 

> 

» 

d. maintaining a computer- 
ized instructional 
management system 

e. implementing the 
overall program 

(To be completed by middle 
school teachers only) 

The services provided with 
the toll-free telephone 
number have been valuable 
to me in 

a. learning to use the 
instructional machines - 

b. instructing pupils in 
using the instructional 
machines 

c. prescribing ins true- ' 
tional strategies 

d. maintaining a computer- 
ized .instructional 
management system 

e. implementing the 
overall program 



2 
2 



3 
3 



3 ' 
3 



4 
4 



4 
4 



5 
5 



5 
5 



5 
5 
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Instructions and ez- 
planationa from printed 
materials furnished by 
the company have been 
valuable to ae in 

a. learning to use 
the instructional 
machines 

b. instructing pupils 
in using the 
instructional 
machines 

o. prescribing instruc- 
tional strategies 

d. maintaining a computer- 
ized instructional 
management system 

e. implementing the 
overall program 

The overall inaervlce 
training has been 
valuable to me;in 

a. learning to use the 
instructional 
machines 

b. instructing pupils 
in using the 
instructional 
machines 

c. prescribing in- 
structional strategies 

d. maintaining a 
computerized instruc- 
tional management 

- system 

e. implementing the 
overall program 



Strongly Strongly 

Diaflgrgfl Dlsaffr** Onrfaafdad, frgcflfl frgEfla 



2 
2 



3 
3 



3 - 



A 



4 
4 



5 
5 
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Pleaae indicate any aapeet^op procedure of inaervioe that you considered 
most helpful this year. 



Please indicate any aspeot or prooedure of inservice that you considered 
least beneficial this year. 



Please Indicate any suggest ions you may have to further improve the 
inservice process for teachers using computers in their programs. 
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Observer 



School 



Date 



Time of Day 
Program 



Oay of Week 



■ Number of Students 
Grade ■ Vest 



Testing Environment, 



Use the following key to rate the conditions of the testing environment. 

P a Poor 

Very Poor - 



VG « Very Good 
G » Good 
A « Acceptable 



VP 



y 



Lighting in the testing area 
Space for each student 
Sound or noise level 
Temperature 

Type^f R. -m: Classroom Library 

Other 

Test Dir ections 

How were the directions given? Read by Proctor 

Other 

1. Audlbleness of the instructions VG 

2. Extent to which proctor provided for 
students' questions VG 

3. The clarity of proctor(s) .answers to 
students' questions VG 

4. Clarity of directions for marking answer VG 

5. Extent to which proctor followed direc- 

. tions in the examiner's manual -VG 

6. Attitude of the proctor toward the 

testing process VG 

7. Accuracy of the procedure for timing 

the test VG 



VG 


G 


•A 


P 


VP 


VG 


G 


A 


P 


VP 












VG 


G 


A 


P 


VP 


VG 


G 


A 


, P 

f 






Lunchroom 


/ 





Written on the Board 



G 

G 

G 
G 

G 

G 

G 



A 

A 

k 
A 

A 

A 
A 



P 

P 

P 
P 

P 

P 

P 



VP 
VP 

VP 

-VP 

VP 
VP 

VP 
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Observer 




Date 


Time of Day 


Day of tfeek lM , \ 


m 

__JIumber of Students 


Proffpan 




« 

Teat 


6 ■ 

1 



Tasting Anvi^m^, 

Use the following key tSJrate the conditions of the teating environment. 



VQ a Very Good 
G a Good 
' A a Acceptable 


* 

P a Poor 
VP a Very Poor 








lighting in the teating area 


VG 


G 


A 


P 


VP 


Space for each student 


VG 


G 


A 


P 


VP 


Sound or noise level 


VG 


G 


A 


P 


VP 


Temperature 


VG 


G 


A 


P 


VP 



Type of Room: Clasaro< 

Othcr_ 

Teat D1—MH,-m. 



Library. 



Lunchroi 



How were the directions given? Read by Proctor, 

Othe r 

1. Audlbleness of the instructions 



_ Written on the Board. 
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2. Extent to which proctor provided for 
students' questions 

3« The clarity of prootor(s) answers to 
students' questions 

4. Clarity of directions for marking answer 

5. Extent to which proctor followed direc- 
tions in the examiner's manual 

6. Attitude of the proctor toward the 
testing process 

7. Accuracy of the procedure for timing 
the test 
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VG 
VG 

VG 

VG 

VG 
VG 

VG 



G 

G 

G 
G 

G 

G 



A 
A 



P 
P 



A 



VP 

VP 

VP 
VP 

VP 

VP 

VP 



QUESTIONS FOR PLL LABS 



Are kids getting accustomed 'to using computer? 



♦ 

r 



Do they seem to be learning more? 



Have you noticed any change 1n attendance since kids started 
using computer? 



Have there been any technical difficulties with the system? 
If so. have these been worked out satisfactorily? 



Have you been satisfied with the services provided by the 
consultant? 



Are you satisfied with -the diagnosis and prescription as provided 
by the computer?^ 



How many minutes per week is average kid on the computer? 
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SCHOOl H ^ GRADE 

USE A NUMBER 2 PENCIL. ERASE COMPLETELY WHEN MAKING CORRECTIONS. 



OOPPPPPPPPPPPPPPPPQPPPOOOOQQQQ OO OOOOC 



OOOOOOOOOOOOOOOOOQOQQOOOO O OOOO OOOOOOL 



[ 



O0OOQO OOP OOOOOOOOOOOOOOQ O O P OQOO Q OQOQC 



OOOOOOOOOOOOOOOOOOOQP O'O PPPQQOOOOOOOQC 



OOOOOOOOOOOOPPPPPPPPPOOOOOOOQ oooobooc 



PPPOPQPOOOOOOOOOOOOOOOOOOOOQQ OOOOOOOC 



t 



OOOOOOOOOOOOOOOOOOOOOOOOOOQO OOOO OOOO L 



rj 



ooooooooooooooooooooooooooo oooooooooi 



qoooqo ooooooooooooooo oooqo ooooooooooc 



O000000000000000000000000000()( H) O o5rtO O ( 

I I I I I I I I I I I I I I I I I I I I f I I I I I I I I I I I I I I I ■ • 



